Moving an object by drag operation on a touch panel

ABSTRACT

A contact state of a finger with respect to a first displayed object is detected. In response to detecting a double-click operation on the first displayed object at a first position in a depthwise direction, the first object is moved onto a surface of a display screen of a 3-D display and the first object is displayed. In response to detecting that the finger has moved from the contact state to a proximity state with respect to the first displayed object, a display position of the first displayed object is changed to a position at which an object nearest in the depthwise direction among displayed objects is displayed.

BACKGROUND

The present invention relates to moving an object by a drag operation ona touch panel where contact and proximity of a finger or the positionsof the finger at contact and proximity may be detected.

Information processing apparatuses equipped with a touch panel haverecently been developed. A touch panel allows the user to intuitivelyoperate information by direct contact with the display.

BRIEF SUMMARY

In accord with certain implementations, a method for moving a displayedobject among a plurality of display layers by an information processingapparatus connected to a three-dimensional (3-D) touch panel capable ofdata communication therewith, involves detecting a contact state of afinger with respect to an object displayed on a first layer; in responseto detecting that the finger has moved from the contact state to aproximity state with respect to the object and in response to the fingerreaching a second layer displayed nearer than the first layer, movingthe object onto the second layer; and displaying the object on thesecond layer.

In certain implementations, the method further involves, in response todetecting that the finger has moved from the proximity state to thecontact state with respect to the object again, displaying the object ona layer with which the finger is in contact at a point on a displaysurface of a display screen where it is detected that the finger hasmoved to the contact state again.

In certain implementations, the method further involves measuring a timefrom when the finger moved to the proximity state with respect to theobject; determining whether a predetermined time has elapsed after astart of the time measurement; in response to determining that thepredetermined time has elapsed, returning the object to a position atwhich the object is displayed at the point on the display surface of thedisplay screen where it is detected that the finger has moved to theproximity state; and displaying the object.

In another implementation, a method for moving an object displayed witha feeling of depth on a three-dimensional (3-D) display by aninformation processing apparatus connected to a 3-D touch panel capableof data communication therewith, involves detecting a contact state of afinger with respect to a first displayed object; in response todetecting a double-click operation on the first displayed object at afirst position in a depthwise direction is detected, moving the firstdisplayed object onto a surface of a display screen of the 3-D displayand displaying the first object; and in response to detecting that thefinger has moved from the contact state to a proximity state withrespect to the first displayed object, changing a display position ofthe first displayed object to a position at which an object nearest inthe depthwise direction among displayed objects is displayed.

In certain implementations, the method further involves, in response todetecting that the finger has moved from the proximity state to thecontact state with respect to the first displayed object again, changingthe display position of the first displayed object to a position on thesurface of the display screen of the 3-D display and displaying thefirst object.

In certain implementations, the method further involves, in response todetecting that the finger has moved again from the proximity state tothe contact state with respect to the first displayed object, changing aposition at which a second object is displayed by a same amount ofmovement as that of the display position of the first displayed object,and displaying the second object.

In another implementation, a method for moving a displayed object amonga plurality of display layers by an information processing apparatusconnected to a two-dimensional (2-D) touch panel and an inclinationangle sensor that detects an angle of inclination of the 2-D touch panelcapable of data communication therewith, involves detecting a contactstate of a finger with respect to an object displayed on a first layer;obtaining the angle of inclination detected by the inclination anglesensor, where the angle of inclination along a display surface at apoint where it is detected that the finger has moved to the contactstate with respect to the object is configured as zero degrees; inresponse to determining that the obtained angle of inclination changesfrom zero degrees to a positive value and in response to a dragging ofthe finger in a higher direction along the angle of inclination, movingthe object onto a second layer displayed nearer than the first layer atthe point where the object reaches the second layer and displaying theobject; and in response to determining that the obtained angle ofinclination changes from zero degrees to a negative value and inresponse to the dragging of the finger to a lower direction along theangle of inclination, moving the object onto a third layer displayeddeeper than the first layer at the point where the object reaches thethird layer and displaying the object.

In certain implementations, the method further involves, in response todetermining that the obtained angle of inclination has returned to zerodegrees again, displaying the object on a layer with which the finger isin contact at the point where the angle of inclination has returned tozero degrees again.

In another implementation, a method for moving an object displayed witha feeling of depth on a three-dimensional (3-D) display by aninformation processing apparatus connected to a two-dimensional (2-D)touch panel and an inclination angle sensor for detecting an angle ofinclination of the 2-D touch panel capable of data communicationtherewith, involves detecting a contact state of a finger with respectto a displayed object; in response to detecting a double-click operationon the displayed object at a first position in a depthwise direction,moving the displayed object onto a surface of a display screen of the3-D display and displaying the object; obtaining the angle ofinclination detected by the inclination angle sensor, where the angle ofinclination along the display surface at a point where the finger hasmoved to the contact state with respect to the displayed object isconfigured as zero degrees; displaying the object inclined at an angleopposite in positive and negative from the obtained angle of inclinationon the 3-D display; and moving the object along the angle of inclinationat the angle opposite in positive and negative from the obtained angleof inclination on the 3-D display in response to detecting a dragging ofthe finger, and displaying the object.

An apparatus, consistent with certain implementations, is connected to athree-dimensional (3-D) touch panel capable of data communicationtherewith for moving a displayed object among a plurality of displaylayers, and the apparatus has a contact-state detection sectionconfigured to detect a contact state of a finger with respect to anobject displayed on a first layer; and an object display sectionconfigured to, in response to a detection that the finger has moved fromthe contact state to a proximity state with respect to the object, andin response to the finger reaching a second layer displayed nearer thanthe first layer, move the object onto the second layer and display theobject on the second layer.

In certain implementations, the object display section is furtherconfigured to, in response to a detection that the finger has moved fromthe proximity state to the contact state with respect to the objectagain, display the object on a layer with which the finger is in contactat a point on a display surface of a display screen where it is detectedthat the finger has moved to the contact state again.

In certain implementations, the apparatus further has a timer sectionconfigured to measure a time from when the finger has moved to theproximity state with respect to the object; a determination sectionconfigured to determine whether a predetermined time has elapsed after astart of the time measurement; and where the object display section isconfigured to, in response to the determination section determining thatthe predetermined time has elapsed, return the object to a position atwhich the object is displayed at the point on the display surface of thedisplay screen where it is detected that the finger has moved to theproximity state and display the object.

An apparatus, consistent with another implementation, is connected to athree-dimensional (3-D) touch panel capable of data communicationtherewith for moving an object displayed with a feeling of depth on a3-D display, and the apparatus has a contact-state detection sectionconfigured to detect a contact state of a finger with respect to a firstdisplayed object; an object display section configured to, in responseto detection of a double-click operation on the first displayed objectat a first position in a depthwise direction, move the first displayedobject onto a surface of a display screen of the 3-D display and displaythe first displayed object; and in response to a detection that thefinger has moved from the contact state to a proximity state withrespect to the first displayed object, change a display position of thefirst displayed object to a position at which an object nearest in thedepthwise direction among displayed objects is displayed.

In certain implementations, the object display section is furtherconfigured to, in response to a determination that the object has movedfrom the proximity state to the contact state again, change the displayposition of the first displayed object to a position on the surface ofthe display screen of the 3-D display and display the object.

In certain implementations, the object display section is furtherconfigured to, in response to a detection that the finger has movedagain from the proximity state to the contact state with respect to thefirst displayed object, change a position at which a second object isdisplayed by a same amount of movement as that of the display positionof the first displayed object and display the second object.

An apparatus, consistent with another implementation, is connected to atwo-dimensional (2-D) touch panel and an inclination angle sensor thatdetects an angle of inclination of the 2-D touch panel capable of datacommunication therewith for moving a displayed object among a pluralityof display layers, and the apparatus has a contact-state detectionsection configured to detect a contact state of a finger with respect toan object displayed on a first layer; an inclination-angle acquisitionsection configured to obtain the angle of inclination detected by theinclination angle sensor, where the angle of inclination along a displaysurface at a point where it is detected that the finger has moved to thecontact state with respect to the object is configured as zero degrees;and an object display section configured to, in response to adetermination that the obtained angle of inclination changes from zerodegrees to a positive value and in response to a dragging of the fingerin a higher direction along the angle of inclination, move the objectonto a second layer displayed nearer than the first layer at a pointwhere the object reaches the second layer and display the object; inresponse to a determination that the obtained angle of inclinationchanges from zero degrees to a negative value and in response to adragging of the finger in a lower direction along the angle ofinclination, move the object onto a third layer displayed deeper thanthe first layer at a point where the object reaches the third layer anddisplay the object.

In certain implementations, the object display section is furtherconfigured to, in response to a determination that the obtained angle ofinclination has returned to zero degrees again, display the object on alayer with which the finger is in contact at the point where the angleof inclination has returned to zero degrees again.

An apparatus, consistent with another implementation, is connected to atwo-dimensional (2-D) touch panel and an inclination angle sensor thatdetects an angle of inclination of the 2-D touch panel capable of datacommunication therewith for moving an object displayed with a feeling ofdepth on a three-dimensional (3-D) display, and the apparatus has acontact-state detection section configured to detect a contact state ofa finger with respect to a displayed object; an object display sectionconfigured to, in response to detection of a double-click operation onthe displayed object at a first position in a depthwise direction, movethe displayed object onto a surface of a display screen of the 3-Ddisplay and display the object; an inclination-angle acquisition sectionconfigured to obtain the angle of inclination detected by theinclination angle sensor, where the angle of inclination along thedisplay surface at a point where the finger has moved to the contactstate with respect to the displayed object is configured as zerodegrees; and wherein the object display section is configured to displaythe object inclined at an angle opposite in positive and negative fromthe obtained angle of inclination on the 3-D display and move the objectalong the angle of inclination at the angle opposite in positive andnegative from the obtained angle of inclination on the 3-D display inresponse to detecting a dragging of the finger, and to display theobject.

In another implementation, a computer program product includes acomputer readable storage medium including computer readable programcode, where the computer readable program code when executed on acomputer causes the computer to detect a contact state of a finger withrespect to an object displayed on a first layer; and, in response todetecting that the finger has moved from the contact state to aproximity state with respect to the object and in response to the fingerreaching a second layer displayed nearer than the first layer, move theobject onto the second layer and display the object on the second layer.

According to the present invention, a desired object may be displayed ata desired relative position without interrupting a series of operations.Furthermore, the visual gap between the position of a finger thatoperates an object and the position of the displayed object may beeasily corrected on a 3-D touch panel that displays a depth, thusallowing the operation without a feeling of difference.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration of an informationprocessing apparatus according to a first example of an implementationof the present subject matter.

FIG. 2 is a block diagram showing a configuration of a 3-D touch panelaccording to the first example implementation of the present subjectmatter.

FIG. 3 is a functional block diagram of the information processingapparatus according to the first example implementation of the presentsubject matter.

FIG. 4 is a flowchart showing the procedure of the CPU of theinformation processing apparatus according to the first exampleimplementation of the present subject matter.

FIG. 5 is a schematic diagram showing the display state of the 3-D touchpanel according to the first example implementation of the presentsubject matter.

FIG. 6 is a schematic diagram showing the moving state of an object on aconventional 3-D touch panel.

FIG. 7(a) and FIG. 7(b) are schematic diagrams showing a drag operationon the 3-D touch panel according to the first example implementation ofthe present subject matter.

FIG. 8 is a schematic diagram showing the movement of an object when aproximity drag on the 3-D touch panel is cancelled according to thefirst example implementation of the present subject matter.

FIG. 9 is a block diagram showing the configuration of an informationprocessing apparatus according to a second example implementation of thepresent subject matter.

FIG. 10 is a flowchart showing the procedure of the CPU of theinformation processing apparatus according to the second exampleimplementation of the present subject matter.

FIG. 11 is a schematic diagram showing the display state of a 3-D touchpanel according to the second example implementation of the presentsubject matter.

FIG. 12 is a schematic diagram showing an operation example of the 3-Dtouch panel 21 according to the second example implementation of thepresent subject matter.

FIG. 13 is an image diagram of object movement by a drag operation onthe 3-D touch panel according to the second example implementation ofthe present subject matter.

FIG. 14 is a block diagram showing the configuration of an informationprocessing apparatus according to a third example implementation of thepresent subject matter.

FIG. 15 is a schematic diagram showing, in outline, a configuration ofan inclination angle sensor of the information processing apparatusaccording to the third example implementation of the present subjectmatter.

FIG. 16 is a functional block diagram of the information processingapparatus 1 according to the third example implementation of the presentsubject matter.

FIG. 17 is a flowchart showing the procedure of the CPU of theinformation processing apparatus according to the third exampleimplementation of the present subject matter.

FIG. 18(a) and FIG. 18(b) are schematic diagrams showing the dragoperation of a 2-D touch panel according to the third exampleimplementation of the present subject matter.

FIG. 19 is a block diagram showing the configuration of an informationprocessing apparatus according to a fourth example implementation of thepresent subject matter.

FIG. 20 is a flowchart showing the procedure of the CPU of theinformation processing apparatus according to the fourth exampleimplementation of the present subject matter.

FIG. 21 is a schematic diagram showing an operation example of the 2-Dtouch panel according to the fourth example implementation of thepresent subject matter.

FIG. 22(a) and FIG. 22(b) are schematic diagrams showing an example ofobject movement on the 2-D touch panel according to the fourth exampleimplementation of the present subject matter.

FIG. 23(a) and FIG. 23(b) are schematic diagrams showing an example ofobject movement on the 2-D touch panel according to the fourth exampleimplementation of the present subject matter.

FIG. 24(a), FIG. 24(b), and FIG. 24(c) are schematic diagrams showing anobject display example on a 3-D display 26 according to the fourthexample implementation of the present subject matter.

DETAILED DESCRIPTION

An apparatus connected to a three-dimensional (3-D) touch panel capableof data communication therewith for moving a displayed object among aplurality of layers or for moving an object displayed with a feeling ofdepth according to embodiments of the present invention will bespecifically described hereinbelow with reference to the drawings. Itshould be noted that the example embodiments below do not limit thepresent subject matter described in the claims and all the combinationsof features described in the embodiments are possible solutions. Theword “Figure” is used below instead of the abbreviation “FIG.” for thedrawings.

It is also to be understood that the present invention may be embodiedin many different forms and should not be limited to the descriptions ofthe example embodiments. The same components are given the samereference designators throughout the examples.

Although the following examples are described when applied to anapparatus in which a computer system implements a computer program, itshould be noted that part of the present subject matter may be embodiedas a computer program that may be executed by a computer. Accordingly,the present subject matter may be embodied as hardware, which is anapparatus connected to a 3-D touch panel, so as to be capable of datacommunication therewith for moving a displayed object among a pluralityof layers or an apparatus for moving an object displayed with a feelingof depth, software, or a combination of hardware and software. Thecomputer program may be stored in any computer-readable recording media,such as a hard disk, a DVD, a CD, an optical storage unit, and amagnetic storage unit.

According to the present subject matter, a desired object may bedisplayed at a desired relative position without interrupting a seriesof operations. Furthermore, a visual gap between a position of a fingerthat operates an object and a position of the displayed object may becorrected on a 3-D touch panel that displays a depth, thus allowingoperation without a feeling of difference.

It should be noted that conception of the present subject matterresulted from recognition of certain limitations associated withconventional touch panel-based devices. For example, it was recognizedthat with conventional touch panel-based devices, a contact buttoncannot be focused or previewed, along with the display size of thebutton. It was further recognized that a display input device for aninput operation detects a proximity state before a finger comes intocontact with a touch panel and that by detecting the proximity state ofthe finger, the focus may be changed depending on the proximity positionto display the button in an enlarged scale. It was additionally observedthat a three-dimensional (3-D) touch panel capable of detecting aproximity state before a finger comes into contact therewith detects theposition of the finger by detecting an electrostatic capacitancegenerated between the touch panel and the finger, thus allowing alight-touch input operation. However, it was recognized that the systemcannot move an object onto an intended layer for an application with alayer structure. Furthermore, it was recognized that even for the sameobject, the operation method differs between a movement by dragging anda movement between layers, and thus the object cannot be moved by aseries of operations with conventional touch panel-based devices. It wasfurther recognized that user of certain applications, such asPowerPoint® developed by Microsoft® and Photoshop® developed by Adobe®,need to select a menu with a right-click operation and then select“Order” item in moving an object to a different layer, which hasproblems in that the operation is complicated and the object cannot bemoved onto a desired layer. Furthermore, it was recognized that with a3-D touch panel combined with a three-dimensional display (3-D display),a visual gap occurs between the position of a finger that operates anobject and the position of the object displayed, which causes a feelingof difference in operation. The present subject matter improves touchpanel-based devices by allowing an object to be moved onto a differentlayer by a series of drag operations by differentiating a drag operationbetween a contact state and a proximity state or by combining adouble-click operation, the inclination of a touch panel, and otherimprovements with the drag operation.

FIG. 1 is a block diagram showing a configuration of an informationprocessing apparatus according to a first example of an implementationof the present subject matter. An information processing apparatus 1includes at least a central processing unit (CPU) 11, a memory 12, astorage device 13, an input/output (I/O) interface 14, a video interface15, a portable-disc drive 16, a communication interface 17, and aninternal bus 18 that connects the foregoing hardware.

The CPU 11 is connected to the foregoing hardware components of theinformation processing apparatus 1 via the internal bus 18. The CPU 11controls the operations of the foregoing hardware components andimplements various software functions in accordance with a computerprogram 100 stored in the storage device 13. The memory 12 is a volatilememory, such as a static random access memory (SRAM) or a synchronousdynamic RAM (SDRAM), in which a load module is expanded when thecomputer program 100 is executed to store temporary data etc. generatedwhen the computer program 100 is executed.

The storage device 13 is a built-in fixed storage device (hard disk), aread-only memory (ROM), or the like. The computer program 100 stored inthe storage device 13 is downloaded using the portable-disc drive 16from a portable recording medium 90, such as a digital versatile disc(DVD) or a compact disc read-only memory (CD-ROM), in which information,such as programs and data, is stored, and at runtime, it is expandedfrom the storage device 13 to the memory 12. Of course, computerprograms downloaded from an external computer (not shown) connected viathe communication interface 17 may be used.

The communication interface 17 is connected to the internal bus 18 andis capable of data transmission and reception to/from the externalcomputer etc. when connected to an external network, such as theInternet, a local area network (LAN), or a wide area network (WAN).

The I/O interface 14 is connected to a 3-D touch panel 21 and acceptsdata entry. The video interface 15 is connected to a 2-D display 22integrated with the 3-D touch panel 21 and displays a given image. Theuser operates an object displayed on the 2-D display 22 with a finger.

FIG. 2 is a block diagram showing a configuration of the 3-D touch panel21 according to the first example implementation of the present subjectmatter. As shown in FIG. 2, the 3-D touch panel 21 includes at least atouch panel section (2-D display) 211, an oscillator circuit 41, adetecting-position control circuit 42, an arithmetic circuit 43, anX-axis input side switch 44, a Y-axis input side switch 45, an X-axisoutput side switch 46, a Y-axis output side switch 47, and a controlcircuit 48.

The touch panel section 211 is a touch panel in which X-axis electrodewires for detecting a position in the X-axis direction and Y-axiselectrode wires for detecting a position in the Y-axis direction arearrayed in a matrix. The oscillator circuit 41 is an oscillator circuitthat generates a pulse signal. The detecting-position control circuit 42is a control circuit for controlling which of the X-axis electrode wiresand the Y-axis electrode wires of the touch panel section 211 are to beconnected.

The arithmetic circuit 43 is a circuit that detects a signal from thetouch panel section 211 and calculates a contact position or a proximityposition of a finger. The X-axis input side switch 44 is a switch forinputting a pulse signal to the input port of the X-axis electrodewires, and the Y-axis input side switch 45 is a switch for inputting apulse signal to the input port of the Y-axis electrode wires. The X-axisoutput side switch 46 is a switch for connecting the output port of theX-axis electrode wires to the arithmetic circuit 43, and the Y-axisoutput side switch 47 is a switch for connecting the output port of theY-axis electrode wires to the arithmetic circuit 43.

The control circuit 48 controls the overall operation of the 3-D touchpanel 21 and instructs the detecting-position control circuit 42 todetect a finger contact position or proximity position, with apredetermined number of electrodes connected. When the position isdetected, the control circuit 48 issues an instruction to individuallyconnect electrodes in the vicinity of the detected position and toconnect the other electrodes by a predetermined number.

The operation of the apparatus connected to the 3-D touch panel 21 withthe above configuration capable of data communication for moving adisplayed object among a plurality of layers will be describedhereinbelow.

FIG. 3 is a functional block diagram of the information processingapparatus 1 according to the first example implementation of the presentsubject matter. In FIG. 3, a contact-state detection section 101 of theinformation processing apparatus 1 receives an output signal from the3-D touch panel 21 and extracts positional information of a displayedobject, layer information, and information on the contact state of thefinger in response to the output signal. The information on the contactstate of the finger is information of whether, for example, the fingeris in a contact state, a proximity state, or a separate state, which maybe determined from output signals from individual sensors of the 3-Dtouch panel 21.

An object display section 102 changes a method for displaying theobject, depending on the information on the contact state of the finger.For example, in the case where it is detected that the finger has movedfrom a contact state to a proximity state with respect to an objectdisplayed on a first layer, the object display section 102 moves theobject onto the second layer and displays it when the finger has reacheda position on a second layer displayed nearer than the first layer.

In the case where it is detected that the finger has moved from theproximity state to the contact state with respect to the object again,the object display section 102 displays the object at the point where itis detected that the finger has moved to the contact state.

A timer section 103 measures the time from the point where the fingerhas moved to the proximity state with respect to the object. Adetermination section 104 determines whether a predetermined time haspassed after the timing is started in the timer section 103. When it isdetermined by the determination section 104 that a predetermined timehas passed after the start of timing, the object display section 102returns the object to the position at which it is displayed at the pointwhere the finger moved to the proximity state and displays it. Here, thepredetermined time is 100 milliseconds (ms), for example, though othertimes are possible.

In the first example, although completion of a drag operation isdetermined on the basis of whether a predetermined time has passed aftermoving to the proximity state, the present subject matter is notparticularly limited thereto. For example, the determination ofcompletion of the drag operation may be made on the basis of whether thedistance from the screen to the finger is longer than a predetermineddistance. In this case, for example, since the resolution in thedirection perpendicular to the screen in a proximity state often has twohundred and fifty six (256) levels of gray, the determination of whetherthe distance from the screen to the finger is longer than apredetermined distance, that is, whether the drag operation has beencompleted, may be made on the basis of whether the resolution is higherthan a predetermined levels of gray.

FIG. 4 is a flowchart showing the procedure of the CPU 11 of theinformation processing apparatus 1 according to the first exampleimplementation of the present subject matter. In FIG. 4, the 3-D touchpanel 21 is provided with the 2-D display 22, and an object is movedbetween at least two layers, that is, a front layer displayed at thefront side and a back layer.

FIG. 5 is a schematic diagram showing the display state of the 3-D touchpanel 21 according to the first example implementation of the presentsubject matter. As shown in FIG. 5, an object 53 is displayed on a backlayer (first layer) 51, and the object 53 is moved onto a front layer(second layer) 52 by dragging the object 53 with a finger (hereinafterreferred to as “ordinary drag”).

FIG. 6 is a schematic diagram showing the moving state of the object 53on a conventional 3-D touch panel. As shown in FIG. 6, the object 53displayed on the back layer 51 is moved onto the front layer 52 by anordinary drag with a finger 61. It should be noted that with theconventional 3-D touch panel, the object 53 cannot be moved onto thefront layer 52, so that the portion where a moved object 62 and thefront layer 52 overlap displays the front layer 52 but does not displaythe moved object 62. The present subject matter allows the object 53 tobe moved onto the front layer (second layer) 52.

Thus, the first example implementation allows a drag operation in aproximity state (hereinafter referred to as “proximity drag”) bydetecting the proximity state of the finger. In other words, in the casewhere the object 53 is moved onto the front layer (second layer) 52during a proximity drag, the moved object 62 is displayed on the frontlayer 52.

Referring back to FIG. 4, the CPU 11 of the information processingapparatus 1 receives an output signal from the 3-D touch panel 21 (stepS401) and determines whether the finger is in a contact state in whichthe finger is in contact with an object displayed on the first layer onthe basis of the received output signal (step S402). If the CPU 11determines that the finger is not in the contact state (step S402: NO),the CPU 11 goes into a waiting state until the finger comes into thecontact state.

If the CPU 11 determines that the finger is in the contact state (stepS402: YES), the CPU 11 determines that an ordinary drag is started anddetermines whether the finger has moved to a proximity state (stepS403). If the CPU 11 determines that the finger has not moved to theproximity state (step S403: NO), the CPU 11 goes into a waiting stateuntil the finger comes into the proximity state.

If the CPU 11 determines that the finger has moved to the proximitystate (step S403: YES), the CPU 11 determines that the finger hasstarted a proximity drag, starts timing with a built-in timer or thelike (step S404), and determines whether a predetermined time has passed(step S405). If the CPU 11 of the information processing apparatus 1determines that a predetermined time has passed (step S405: YES), theCPU 11 returns the finger to the position where the object is displayedat the point where the finger has moved to the proximity state anddisplays the object (step S406). If the CPU 11 determines that apredetermined time has not passed (step S405: NO), the CPU 11 determineswhether the finger has performed the proximity drag to a position wherethe second layer is displayed (step S407). If the CPU 11 determines thatthe finger has not performed the proximity drag to the position wherethe second layer is displayed (step S407: NO), the CPU 11 goes to awaiting state.

If the CPU 11 determines that the finger has performed the proximitydrag to the position where the second layer is displayed (step S407:YES), the CPU 11 displays the object on the second layer (step S408) anddetermines whether the finger has moved to the contact state again (stepS409). If the CPU 11 determines that the finger has not moved again(step S409: NO), the CPU 11 goes to a moving waiting state again.

If the CPU 11 determines that the finger has moved again (step S409:YES), the CPU 11 determines that the finger has returned to the ordinarydrag and displays the object on a layer with which the finger is incontact when it is detected that the finger has moved to the contactstate (step S410).

FIG. 7(a) and FIG. 7(b) are schematic diagrams showing a drag operationon the 3-D touch panel 21 according to the first example implementationof the present subject matter. FIG. 7(a) is an image diagram of objectmovement through a drag operation on the 3-D touch panel 21, and FIG.7(b) is an image diagram of an actual drag operation on the 3-D touchpanel 21.

As shown in FIG. 7(a), an ordinary drag is first performed in a section71 to move the object 53 on a lower layer (first layer) 75.Subsequently, a proximity drag is performed in a section 72. When thefinger comes near (has reached) the upper layer (second layer) 74 duringa proximity drag, the object 53 is moved onto the upper layer (secondlayer) 74. If the finger returns to the ordinary drag in a section 73,the object 53 is moved on the upper layer (second layer) 74.

In the case where the object 53 first moves on the upper layer 74 andcomes near (has reached) the lower layer 75 during a proximity drag, theobject 53 is moved onto the lower layer 75, contrary to FIG. 7(a). Whenthe finger returns to the ordinary drag, the object 53 is moved onto thelower layer 75.

The actual movement of the user's finger is shown in FIG. 7(b). As shownin FIG. 7(b), the user first touches the target object with the fingerand moves the object with an ordinary drag without separating the fingerfrom the surface (first layer) of the 3-D touch panel 21 (section 71).Next, in the case where the finger approaches a second layer, and theuser wants to move the object to the approaching layer, the userseparates the finger from the surface of the 3-D touch panel 21 andperforms a proximity drag (section 72). The user touches the surface ofthe 3-D touch panel 21 on the second layer with the finger again toreturn to the ordinary drag, and thus, the object may be moved on thesecond layer (section 73).

Referring back to FIG. 4, as described above, if the CPU 11 of theinformation processing apparatus 1 determines that a predetermined timehas passed (step S405: YES), the CPU 11 returns the finger to theposition where the object is displayed at the point where the finger hasmoved to the proximity state and displays the object (step S406).

FIG. 8 is a schematic diagram showing the movement of the object whenthe proximity drag on the 3-D touch panel 21 is cancelled according tothe first example implementation of the present subject matter. First,the object 53 is moved on the back layer 51 by an ordinary drag, and atthe point where the object 53 has moved to a position 81, a proximitydrag is started. In this case, if the finger has not moved from theproximity state to the contact state during a predetermined time, thatis, if the finger has not returned from the proximity drag to theordinary drag, the CPU 11 determines that the drag is not the proximitydrag and returns the object 53 to the position 81 where it is determinedthat the finger has moved to the proximity state and displays the object53.

As described above, the first example implementation allows a desiredobject to be moved so as to be displayed on a desired layer withoutinterrupting a series of operations by combining a proximity drag withone or more ordinary drags.

FIG. 9 is a block diagram showing the configuration of an informationprocessing apparatus 1 according to a second example implementation ofthe present subject matter. Since the configuration of the informationprocessing apparatus 1 according to the second example implementation isthe same as that of the first example implementation, detaileddescription thereof will be omitted by giving the same referencedesignators. The second example implementation differs from the firstexample implementation in that an object is moved on a display screenwith a feeling of depth by a drag operation by using a 3-D display.

The I/O interface 14 is connected to the 3-D touch panel 21 and acceptsdata entry as in the first example. On the other hand, unlike the firstexample, the video interface 15 is connected to a 3-D display 23integrated with the 3-D touch panel 21 and displays a 3-D image with afeeling of depth. The user operates an object displayed on the 3-Ddisplay 23 with a finger.

The operation of the information processing apparatus 1 connected to the3-D touch panel 21 with the above configuration capable of datacommunication for moving a displayed object will be describedhereinbelow. Although the functional block diagram of the informationprocessing apparatus 1 according to the second example implementation isthe same as that of the first example, the functions of thecontact-state detection section 101 and the object display section 102differ.

Specifically, the contact-state detection section 101 of the informationprocessing apparatus 1 receives an output signal from the 3-D touchpanel 21 and extracts positional information of a displayed object,depth information, and information on the contact state of the finger inresponse to the output signal. The depth information is information onthe depthwise position of an object displayed on the 3-D display 23, andthe information on the contact state of the finger includes informationof whether a double-click operation has been performed, in addition toinformation of whether the finger is in a contact state, a proximitystate, or a separate state, for example.

The object display section 102 changes a method for displaying anobject, depending on the information on the contact state of the finger.For example, in the case where a double-click operation is detected, theobject display section 102 moves a first object to the surface of thedisplay screen of the 3-D display 23 and displays it in whicheverposition in the depthwise direction the first object is displayed. Inthe case where it is detected that the finger has moved from the contactstate to a proximity state with respect to the first object displayed onthe surface of the display screen of the 3-D display 23, the objectdisplay section 102 moves the display position of the first object to aposition at which a second object is displayed on the nearest side inthe depthwise direction of the displayed objects and displays the firstobject.

In the case where it is detected that the finger has moved from theproximity state to the contact state again, the object display section102 changes the display position of the first object onto the surface ofthe display screen of the 3-D display 23 and displays the first object.At the same time, the object display section 102 changes the depthwisedisplay position of the second object by the amount of depthwisemovement of the first object. Thus, with the ordinary drag, the targetobject may always be displayed on the surface of the display screen ofthe 3-D display 23, and with the proximity drag, the object may bedisplayed at the display position of the nearest object in the depthwisedirection. This may thus reduce a visual feeling of difference betweenthe actual sense of operation with the finger and the depthwise positionof an object displayed on the display screen.

FIG. 10 is a flowchart showing the procedure of the CPU 11 of theinformation processing apparatus 1 according to the second exampleimplementation of the present subject matter. In FIG. 10, the 3-D touchpanel 21 is provided with the 3-D display 23, in which a feeling ofdepth is provided. The object is moved while the feeling of visualdifference between the depthwise position of the displayed object andthe feeling of actual finger operation is reduced.

FIG. 11 is a schematic diagram showing the display state of the 3-Dtouch panel 21 according to the second example implementation of thepresent subject matter. As shown in FIG. 11, assuming that the depthwisedirection of the 3-D touch panel 21 is the Z-axis direction, the fingeris always on a surface 111 of the display screen of the 3-D display 23,while displayed objects 112 and 113 differ in position in the Z-axisdirection. Accordingly, for example, the object 112 may be operatedwithout a touch with the finger, while the object 113 cannot be operatedunless the finger goes therethrough.

Thus, the second example implementation is configured such that a targetobject is moved to the surface 111 of the display screen of the 3-Ddisplay 23 by detecting a double-click operation with a finger. Thisallows the user to operate a target object while visually checking whichobject is to be operated.

Referring back to FIG. 10, the CPU 11 of the information processingapparatus 1 receives an output signal from the 3-D touch panel 21 (stepS1001) and determines whether the displayed first object has beendouble-clicked with a finger (step S1002).

In the case where the CPU 11 determines that no double-click operationhas been performed (step S1002: NO), the CPU 11 goes into a double-clickoperation waiting state. If the CPU 11 determines that a double-clickoperation has been performed (step S1002: YES), the CPU 11 determinesthat the finger is in an ordinary drag operation and moves the firstobject to the surface of the display screen of the 3-D display 23 anddisplays it (step S1003).

FIG. 12 is a schematic diagram showing an operation example of the 3-Dtouch panel 21 according to the second example of the present subjectmatter. As shown in FIG. 12, assuming that the depthwise direction ofthe 3-D touch panel 21 is the Z-axis direction, the object 112, whichmay be operated without a touch of a finger in FIG. 11, is moved to thesurface 111 of the display screen of the 3-D display 23 bydouble-clicking thereon, so that the object 112 may be operated with thefinger with a feeling of direct operation.

Likewise, the object 113, which cannot be operated unless the fingerpasses therethrough, is moved to the surface 111 of the display screenof the 3-D display 23 by double-clicking thereon, so that the object 113may be operated with the finger with a feeling of direct operation.

Referring back to FIG. 10, the CPU 11 of the information processingapparatus 1 determines that the finger is in contact with the displayedfirst object and determines whether the finger has moved to a proximitystate (step S1004). If the CPU 11 determines that the finger has notmoved to a proximity state (step S1004: NO), the CPU 11 goes into aproximity-state waiting state. If the CPU 11 determines that the fingerhas moved to the proximity state (step S1004: YES), the CPU 11determines that a proximity drag has been started, displays the firstobject to be subjected to the proximity drag at the same depthwiseposition, that is, the same Z-axis direction, as that of the nearestside second object (step S1005), and determines whether the finger hasmoved to the contact state again (step S1006). If the CPU 11 determinesthat the finger has not moved again (step S1006: NO), the CPU 11 goesinto a movement waiting state again.

If the CPU 11 determines that the finger has moved again (step S1006:YES), the CPU 11 determines that the finger has returned to the ordinarydrag and moves the first object to the surface of the display screen ofthe 3-D display 23 and displays it (step S1007). The CPU 11 also movesthe second object by the same amount of distance as in step S1007 anddisplays it (step S1008).

FIG. 13 is an image diagram of object movement by a drag operation onthe 3-D touch panel 21 according to the second example implementation ofthe present subject matter. First, in a section 121, an ordinary drag isperformed so that the object 112 is moved at a position deeper than thesurface 111 of the display screen of the 3-D display 23. The ordinarydrag is performed after the object 112 is moved to the surface 111 ofthe display screen of the 3-D display 23 by a double-clicking operationand is displayed.

Subsequently, in a section 122, a proximity drag is performed. By theproximity drag, the object 112 is displayed at the same depthwiseposition, that is, the same Z-axis direction position, as that of theobject 113 displayed at the nearest side. Accordingly, this may providethe user with a feeling of operation that the object 112 is raised tothe near side as the user separates (raises) the finger to the nearside. When the finger returns to the ordinary drag in a section 123, inother words, the user brings the finger into contact with the surface111 of the display screen of the 3-D display 23 again, the object 112 ismoved from the depthwise position as that of the nearest side object 113to the surface 111 of the display screen of the 3-D display 23 and isdisplayed.

At that time, the second object 113 other than the object 112 is movedby the amount delta-Z (ΔZ) of movement from the depthwise position ofthe nearest side object 113 to the surface 111 of the 3-D display 23.This may prevent generation of difference in the feeling of distance inthe depthwise direction due to object movement by the operation.

As described above, the second example implementation allows a feelingof operation on a desired object and a feeling of actual fingeroperation to be naturally linked without interrupting a series ofoperations by combining a double-click operation and a proximity drag.

FIG. 14 is a block diagram showing the configuration of an informationprocessing apparatus 1 according to a third example implementation ofthe present subject matter. Since the configuration of the informationprocessing apparatus 1 is the same as that of the first example,detailed description thereof will be omitted by giving the samereference designators. The third example implementation differs from thefirst and second examples in that a 2-D display and an inclination anglesensor are used so that a desired object is moved by a drag operation tobe displayed on a desired layer without interrupting a series ofoperations.

In FIG. 14, the I/O interface 14 is connected to a 2-D touch panel 24and accepts data entry. Unlike the first example, the video interface 15is connected to a 2-D display 25 integrated with the 2-D touch panel 24and displays a given image. The user operates an object displayed on the2-D display 25 with a finger.

Furthermore, an inclination angle sensor 30 is connected to the internalbus 18, so that the angle of inclination of the 2-D touch panel 24during a drag operation may be detected. An object subjected to a dragoperation may be moved and displayed onto a desired layer in accordancewith the detected angle of inclination.

The inclination angle sensor 30 is a kind of acceleration sensor and isan inertia sensor for measuring acceleration. In the third exampleimplementation, a capacitance acceleration sensor is used. It isunderstood that a sensor using another detection principle may be used.

FIG. 15 is a schematic diagram showing, in outline, a configuration ofthe inclination angle sensor 30 of the information processing apparatus1 according to the third example implementation of the present subjectmatter. As shown in FIG. 15, the inclination angle sensor 30 includes amoving element 301, four springs 302 that limit the movement of themoving element 301, comb-like movable electrodes 304 for causing changesin electrostatic capacitance depending on the movement (moving distance)of the moving element 301, and fixed electrodes 303. The unit cells ofthe electrodes are formed such that one movable electrode 304 issandwiched between two fixed electrodes 303.

In the case where opposite-phase clock signals are applied to two fixedelectrodes 303 of a unit cell and when the movable electrode 304 comesclose to either of the fixed electrodes 303 due to acceleration, achange in electric charge, with the same phase as that of a clock signalapplied to the approached fixed electrode 303, occurs in the movableelectrode 304. By amplifying the change in electric charge to performsynchronous detection and rectification, voltage output proportional tothe moving distance of the moving element 301, that is, theacceleration, is obtained. In the third example implementation, gravityis measured as the acceleration to calculate the inclination of theinclination angle sensor 30.

The operation of the apparatus connected to the 2-D touch panel 24 withthe above configuration capable of data communication for moving adisplayed object among a plurality of layers will be describedhereinbelow.

FIG. 16 is a functional block diagram of the information processingapparatus 1 according to the third example of the present subjectmatter. In FIG. 16, the contact-state detection section 101 of theinformation processing apparatus 1 receives an output signal from the2-D touch panel 24 and extracts positional information of a displayedobject, layer information, and information on the contact state of thefinger in response to the output signal. The information on the contactstate of the finger is information of whether, for example, the fingeris in a contact state, a proximity state, or a separate state, which maybe determined from output signals from individual sensors of the 2-Dtouch panel 24.

The inclination-angle acquisition section 105 receives an output signalfrom the inclination angle sensor 30 to obtain the angle of inclination.

The inclination angle sensor 30 detects an angle of inclination withrespect to an angle of inclination, configured as zero degrees, at thepoint where the finger touches the 2-D touch panel 24 (at the pointwhere the finger comes into contact with one object) to determinewhether the angle is positive or negative. This allows a desired objectto be moved onto a desired layer and to be displayed thereon even if the2-D touch panel 24 is inclined from the beginning.

The object display section 102 changes a method for displaying anobject, depending on the information on the contact state of the fingerand the angle of inclination. For example, in the case where the fingeris in contact with an object displayed on a first layer, if a positiveangle of inclination is obtained, the object display section 102 movesthe object onto a second layer displayed nearer than the first layer anddisplays it when the object reaches a position where the second layer isdisplayed.

In contrast, in the case where the finger is in a contact state withrespect to the object displayed on the first layer, if a negative angleof inclination is obtained, the object display section 102 moves theobject onto a third layer displayed deeper than the first layer anddisplays it when the object reaches the third layer.

FIG. 17 is a flowchart showing the procedure of the CPU 11 of theinformation processing apparatus 1 according to the third exampleimplementation of the present subject matter. In FIG. 17, the 2-D touchpanel 24 is provided with the 2-D display 25, and an object is movedbetween at least two layers, that is, a front layer displayed at thefront side and a back layer.

As shown in FIG. 17, the CPU 11 of the information processing apparatus1 receives an output signal from the 2-D touch panel 24 (step S1701) anddetermines whether the finger is in contact with an object displayed onthe first layer on the basis of the received output signal (step S1702).If the CPU 11 determines that the finger is not in the contact state(step S1702: NO), the CPU 11 goes into a waiting state until the fingercomes into the contact state.

If the CPU 11 determines that the finger is in the contact state (stepS1702: YES), the CPU 11 determines that an ordinary drag is started andobtains an angle of inclination from the inclination angle sensor 30(step S1703). The CPU 11 determines whether the acquired angle ofinclination is positive (step S1704), in which case if it is determinedthat the angle of inclination is positive (step S1704: YES), the CPU 11displays the object on a second layer displayed nearer than the firstlayer when the object has reached a position at which the second layeris displayed (step S1705).

If the CPU 11 determines that the angle of inclination is not positive(step S1704: NO), the CPU 11 displays the object on a third layerdisplayed deeper than the first layer when the finger has reached aposition at which the third layer is displayed (step S1706). The CPU 11then determines whether the angle of inclination has returned to zero(0) (step S1707).

If the CPU 11 determines that the angle of inclination has not returnedto zero (0) (step S1707: NO), the CPU 11 determines whether the fingerhas separated from the surface of the 2-D touch panel 24 (step S1708).If the CPU 11 determines that the finger has not separated (step S1708:NO), the CPU 11 returns the process to step S1707 and repeats the aboveprocesses. If the CPU 11 determines that the angle of inclination hasreturned to zero (0) (step S1707: YES), or if CPU 11 determines that thefinger has separated (step S1708: YES), the CPU 11 displays the objecton a layer that is in contact at the point where the CPU 11 determinesthat angle of inclination has returned to zero (0) again (step S1709).

FIG. 18(a) and FIG. 18(b) are schematic diagrams showing the dragoperation of the 2-D touch panel 24 according to the third exampleimplementation of the present subject matter. FIG. 18(a) is an imagediagram of object movement by a drag operation on the 2-D touch panel24, and FIG. 18(b) is an image diagram of an actual drag operation onthe 2-D touch panel 24.

As shown in FIG. 18(a), an ordinary drag is first performed in a section181 to move an object 183 on a lower layer (first layer) 185.Subsequently, an ordinary drag is performed in a section 182, with the2-D touch panel 24 itself inclined. When the finger comes near (reaches)an upper layer (second layer) 184, the object 183 moves onto the upperlayer (second layer) 184 because the angle of inclination is positive.If the angle of inclination is negative, the object 183 moves onto athird layer (not shown) deeper than the lower layer 185 when the fingercomes near the third layer.

The actual movement of the user's finger is shown in FIG. 18(b). Asshown in FIG. 18(b), the user first touches the target object with thefinger and moves the object with an ordinary drag without separating thefinger from the surface of the 2-D touch panel 24 (section 181). Next,in the case where the finger approaches another layer, and the userwants to move the object to the approaching layer, the user performs anordinary drag while inclining the 2-D touch panel 24 to this side(section 182). Thus, by actually inclining the 2-D touch panel 24, adesired object may be moved onto a desired layer.

In the case where information of whether, for example, the finger is ina contact state, a proximity state, or a separate state, is obtained asan output signal from the 2-D touch panel 24, the distance between thefinger and the 2-D touch panel 24 may also be estimated from a change inelectrostatic voltage, and thus, the angle of inclination of the motionof the finger may also be estimated. Accordingly, the same advantagesmay be offered by inclining the motion of the finger and not byinclining the 2-D touch panel 24.

As described above, the third example implementation allows a desiredobject to be moved to be displayed on a desired layer by detecting theangle of inclination without interrupting a series of operations.

FIG. 19 is a block diagram showing the configuration of an informationprocessing apparatus 1 according to a fourth example implementation ofthe present subject matter. Since the configuration of the informationprocessing apparatus 1 is the same as that of the first example,detailed description thereof will be omitted by giving the samereference designators. The fourth example implementation differs fromthe first to third examples in that an object is moved at differentheights on a display screen with a feeling of depth by a drag operationby using a 3-D display and the inclination angle sensor 30.

In FIG. 19, the I/O interface 14 is connected to the 2-D touch panel 24and accepts data entry. Unlike the first example, the video interface 15is connected to a 3-D display 26 integrated with the 2-D touch panel 24and displays a 3-D image with a feeling of depth. The user operates anobject displayed on the 3-D display 26 with a finger.

Furthermore, the inclination angle sensor 30 is connected to theinternal bus 18, so that the angle of inclination during a dragoperation may be detected. An object subjected to a drag operation maybe moved to a desired depthwise position in accordance with the angle ofinclination and may be displayed.

The operation of the information processing apparatus 1 connected to the2-D touch panel 24 with the above configuration capable of datacommunication for moving a displayed object will be describedhereinbelow. Although the functional block diagram of the informationprocessing apparatus 1 according to the fourth example implementation isthe same as that of the third example implementation, the functions ofthe contact-state detection section 101 and the object display section102 differ.

Specifically, the contact-state detection section 101 of the informationprocessing apparatus 1 receives an output signal from the 2-D touchpanel 24 and extracts positional information of a displayed object,depth information, and information on the contact state of the finger inresponse to the output signal. The depth information is information onthe depthwise position of an object displayed on the 3-D display 26, andthe information on the contact state of the finger includes informationon whether a double-click operation has been performed, in addition tothe information of whether, for example, the finger is in a contactstate, a proximity state, or a separate state.

The inclination-angle acquisition section 105 receives an output signalfrom the inclination angle sensor 30 to obtain the angle of inclination.

The inclination angle sensor 30 detects an angle of inclination, withrespect to an angle of inclination configured as zero degrees, at thepoint where the finger comes into contact with the 2-D touch panel 24 todetermine whether the angle is positive or negative. This allows atarget object to be moved to a desired depthwise position and bedisplayed thereon even if the 2-D touch panel 24 is inclined from thebeginning.

The object display section 102 changes a method for displaying theobject, depending on the information on the contact state of the fingerand the angle of inclination. For example, in the case where a doubleclick operation is detected, the target object is moved to the surfaceof the display screen of the 3-D display 26 at whichever depthwiseposition the target object is displayed. For example, in the case wherethe finger is in contact with an object displayed on a first layer, if apositive angle of inclination is obtained, the object is inclined at anangle opposite in positive and negative to the obtained angle ofinclination. Accordingly, even if the 2-D touch panel 24 is inclined,the object displayed on the 3-D display 26 does not differ from theinitial display state for the user.

FIG. 20 is a flowchart showing the procedure of the CPU 11 of theinformation processing apparatus 1 according to the fourth exampleimplementation of the present subject matter. In FIG. 20, the 2-D touchpanel 24 is provided with the 3-D display 26. A displayed object ismoved while a visual feeling of difference between the depthwiseposition of the object and the actual sense of operation with the fingeris reduced.

As shown in FIG. 20, the CPU 11 of the information processing apparatus1 receives an output signal from the 2-D touch panel 24 (step S2001) anddetermines whether a double-click operation with a finger has beenperformed on the displayed object on the basis of the received outputsignal (step S2002). If the CPU 11 determines that no double-clickoperation has been performed (step S2002: NO), the CPU 11 goes into anoperation waiting state.

If the CPU 11 determines that a double-click operation has beenperformed (step S2002: YES), the CPU 11 determines to perform anordinary drag and moves the object to the surface of the display screenof the 3-D display 26 and displays it (step S2003).

FIG. 21 is a schematic diagram showing an operation example of the 2-Dtouch panel 24 according to the fourth example implementation of thepresent subject matter. As shown in FIG. 21, assuming that the depthwisedirection of the 2-D touch panel 24 is the Z-axis direction, the object112, which may be operated without a touch of a finger in FIG. 11, ismoved to the surface 111 of the display screen of the 3-D display 26 bydouble-clicking thereon, so that the object 112 may be operated with thefinger with a feeling of direct operation.

Referring back to FIG. 20, the CPU 11 of the information processingapparatus 1 determines that a ordinary drag is started and obtains anangle of inclination from the inclination angle sensor 30 (step S2004).The CPU 11 displays the object inclined at an angle opposite in positiveand negative to the obtained angle of inclination (step S2005) andcontinues the ordinary drag. The CPU 11 moves the object along theobtained inclination and displays it (step S2006).

FIG. 22(a) and FIG. 22(b), and FIG. 23(a) and FIG. 23(b), are schematicdiagrams showing examples of object movement on the 2-D touch panel 24according to the fourth example implementation of the present subjectmatter. When the 2-D touch panel 24 is inclined, the surface 111 of thedisplay screen of the 3-D display 26 is also inclined. If the angle ofinclination theta is greater than zero (θ>0) (i.e., positive), as shownin FIG. 22(b), the object 112 is moved along the surface 111 of thedisplay screen of the 3-D display 26 by an ordinary drag, and when theobject 112 reaches a layer nearer than the original layer 221 located onthe back, that is, a front layer 222, as shown in FIG. 22(a), the object112 is displayed as moved onto the front layer 222. If the angle ofinclination theta is less than zero (θ<0) (i.e., negative), as shown inFIG. 23(b), the object 112 is moved along the surface 111 of the displayscreen of the 3-D display 26 by an ordinary drag, and when the object112 reaches a layer deeper than the original layer located on the back,that is, a back layer 223, as shown in FIG. 23(a), the object 112 isdisplayed as moved onto the back layer 223.

FIG. 24(a), FIG. 24(b), and FIG. 24(c) are schematic diagrams showing anobject display example on the 3-D display 26 according to the fourthexample implementation of the present subject matter. As shown in FIG.24(a), a 3-D model object 241 is displayed under the 3-D display 26.

When the 3-D display 26 is inclined at the angle theta (θ) in thisstate, the object 241 may also be generally inclined by the angle theta(θ), as shown in FIG. 24(b), so that the positional relationship betweenthe surface of the 3-D display 26 and the object 241 does not change.

However, in the fourth example implementation, when the 3-D display 26is inclined at the angle theta (θ), relative display coordinates of theobject 241 are inclined at an angle negative theta (−θ) opposite inpositive and negative from the angle of inclination theta (θ)calculated, so that the object 241 is displayed on the 3-D display 26,as shown in FIG. 24(c). Accordingly, even if the 3-D display 26 isinclined at the angle theta (θ), the object 241 may be viewed by theuser as if it is displayed in the initial state without inclination.

Accordingly, in the case where the 2-D touch panel 24 is inclined, theobject 241 to be dragged on the 3-D display 26 is viewed from the useras if it is moved directly in the heightwise direction of the object241, thus reducing a visual feeling of difference.

As described above, the fourth example allows a feeling of operation ona desired object and a feeling of actual finger operation to benaturally linked without interrupting a series of operations bycombining a double-click operation and the inclination of the touchpanel.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), a portablecompact disc read-only memory (CD-ROM), an optical storage device, amagnetic storage device, or any suitable combination of the foregoing.In the context of this document, a computer readable storage medium maybe any tangible medium that can contain, or store a program for use byor in connection with an instruction execution system, apparatus, ordevice.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as JAVA™, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention have been described with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in acomputer-readable storage medium that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablestorage medium produce an article of manufacture including instructionswhich implement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

A data processing system suitable for storing and/or executing programcode will include at least one processor coupled directly or indirectlyto memory elements through a system bus. The memory elements can includelocal memory employed during actual execution of the program code, bulkstorage, and cache memories which provide temporary storage of at leastsome program code in order to reduce the number of times code must beretrieved from bulk storage during execution.

Input/output or I/O devices (including but not limited to keyboards,displays, pointing devices, etc.) can be coupled to the system eitherdirectly or through intervening I/O controllers.

Network adapters may also be coupled to the system to enable the dataprocessing system to become coupled to other data processing systems orremote printers or storage devices through intervening private or publicnetworks. Modems, cable modems and Ethernet cards are just a few of thecurrently available types of network adapters.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a,” “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

It is to be understood that the present subject matter is not limited tothe foregoing examples and various changes and modifications may be madewithin the spirit of the present subject matter.

What is claimed is:
 1. A method for moving an object displayed with afeeling of depth on a three-dimensional (3-D) display by an informationprocessing apparatus connected to a 3-D touch panel capable of datacommunication therewith, comprising: detecting a contact state of afinger with respect to a first displayed object; in response todetecting a double-click operation on the first displayed object at afirst position in a depthwise direction, moving the first displayedobject onto a surface of a display screen of the 3-D display anddisplaying the first object; in response to detecting that the fingerhas moved from the contact state to a proximity state with respect tothe first displayed object, changing a display position of the firstdisplayed object from the surface of the display screen to a position atwhich an object nearest in the depthwise direction among displayedobjects is and remains displayed; and in response to detecting that thefinger has moved from the proximity state to the contact state withrespect to the first displayed object again, changing the displayposition of the first displayed object to a position on the surface ofthe display screen of the 3-D display and displaying the first object.2. The method according to claim 1, further comprising, in furtherresponse to detecting that the finger has moved from the proximity stateto the contact state with respect to the first displayed object again,changing a position at which a second object is displayed by a sameamount of movement as that of the display position of the firstdisplayed object and displaying the second object.
 3. An apparatusconnected to a three-dimensional (3-D) touch panel capable of datacommunication therewith for moving an object displayed with a feeling ofdepth on a 3-D display, the apparatus comprising: a contact-statedetection section configured to detect a contact state of a finger withrespect to a first displayed object; and an object display sectionconfigured to: in response to detection of a double-click operation onthe first displayed object at a first position in a depthwise direction,move the first displayed object onto a surface of a display screen ofthe 3-D display and display the first displayed object; in response todetection that the finger has moved from the contact state to aproximity state with respect to the first displayed object, change adisplay position of the first displayed object from the surface of thedisplay screen to a position at which an object nearest in the depthwisedirection among displayed objects is and remains displayed; and inresponse to detection that the finger has moved from the proximity stateto the contact state with respect to the first displayed object again,change the display position of the first displayed object to a positionon the surface of the display screen of the 3-D display and display theobject.
 4. The apparatus according to claim 3, where the object displaysection is further configured to, in further response to detection thatthe finger has moved from the proximity state to the contact state withrespect to the first displayed object again, change a position at whicha second object is displayed by a same amount of movement as that of thedisplay position of the first displayed object and display the secondobject.
 5. The method according to claim 1, further comprising, inresponse to detecting that the finger in the contact state has moved onthe surface of the display screen, moving the first displayed object onthe surface of the display screen and moving a second displayed objectby a same amount of movement as the movement of the first displayedobject.
 6. The apparatus according to claim 3, where the object displaysection is further configured to, in response to detection that thefinger in the contact state has moved on the surface of the displayscreen, move the first displayed object on the surface of the displayscreen and move a second displayed object by a same amount of movementas the movement of the first displayed object.
 7. A computer programproduct comprising a computer readable storage medium comprisingcomputer readable program code for moving an object displayed with afeeling of depth on a three-dimensional (3-D) display by an informationprocessing apparatus connected to a 3-D touch panel capable of datacommunication therewith, where the computer readable program code whenexecuted on a computer causes the computer to: detect a contact state ofa finger with respect to a first displayed object; in response todetecting a double-click operation on the first displayed object at afirst position in a depthwise direction, move the first displayed objectonto a surface of a display screen of the 3-D display and display thefirst object; in response to detecting that the finger has moved fromthe contact state to a proximity state with respect to the firstdisplayed object, change a display position of the first displayedobject from the surface of the display screen to a position at which anobject nearest in the depthwise direction among displayed objects is andremains displayed; and in response to detecting that the finger hasmoved from the proximity state to the contact state with respect to thefirst displayed object again, change the display position of the firstdisplayed object to a position on the surface of the display screen ofthe 3-D display and display the object.
 8. The computer program productaccording to claim 7, where the computer readable program code whenexecuted on a computer further causes the computer to, in furtherresponse to detecting that the finger has moved from the proximity stateto the contact state with respect to the first displayed object again,change a position at which a second object is displayed by a same amountof movement as that of the display position of the first displayedobject and display the second object.
 9. The computer program productaccording to claim 7, where the computer readable program code whenexecuted on a computer further causes the computer to, in response todetecting that the finger in the contact state has moved on the surfaceof the display screen, move the first displayed object on the surface ofthe display screen and move a second displayed object by a same amountof movement as the movement of the first displayed object.